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2  Brief  Summary  of  Technical  Resutts 

2.1  ^iplying  tile  Contact  ABgration  Policy  to  Phoenix 

The  contact  migration  policy  (Grupen  1991)  is  a  reactive  approach  to  constraining 
(grasping)  a  rigid  body  that  incrementally  constructs  a  stationary  process  model  of  the 
body  in  order  to  refine  the  grasp.  The  algorithm  employed  has  many  of  the  character¬ 
istics  of  an  anytime  algorithm  (Dean  and  Boddy  1988),  and  is  therefore  an  attractive 
candidate  for  real-time  applications  such  as  positioning  bulldozers  in  the  Phoenix 
fire-fifi^ting  simulation.  However,  we  have  found  that  several  of  the  assumptions 
underlying  this  algorithm  do  not  hold  in  the  Phoenix  environment,  so  that  such  a 
purely  reactive  control  medianism  is  inadequate  in  Phoenix  (see  Section  3.1).  Based 
on  these  findings,  and  on  a  broader  survey  of  the  application  of  reactivity  in  robotic 
systems,  we  argue  for  the  mixtmre  of  reactive  and  deliberative  (or  predictive)  mecha¬ 
nisms  to  coordinate  the  fire-fighting  activities  of  Phoenix  agents.  The  survey  on  reac¬ 
tivity  has  led  to  the  development  of  a  graduate  seminar  on  reactive  robotic  systems 
being  offered  this  folL 

2J2  Resolving  the  Ddfiberative/Eteactive  IMlemma 

We  pursued  this  argument  -  that  the  control  architecture  in  Phoenix  should  combine 
deliberative  and  reactive  control  -  by  following  several  related  lines  of  research.  In 
the  first,  we  explored  the  effects  of  pladng  deliberative  and  reactive  controllers  on 
separate  processors  within  the  same  architecture  (as  in  Phoenix).  We  derived  a  sim¬ 
ple  mathematical  model  of  the  interaction  of  two  tasks  running  concurrently  on  a 
uniprocessor.  We  modeled  the  rate  at  which  two  periodic  tasks  would  interrupt  each 
other.  From  the  model,  we  argued  that  placing  reflexive  control  and  cognitive  control 
on  different  processors  is  not  justified  by  the  different  time-scales  over  which  they  act, 
but  is  justified  by  the  extent  to  which  the  higher  priority  task  dominates  the 
uniprocessor. 

The  second  line  of  research  addressed  the  question,  when  should  an  agent  deliberate 
and  when  should  it  react?  We  have  attempted  to  answer  this  question  by  establishing 
a  set  of  principles  and  criteria  that  govern  this  decision.  To  evaluate  these  guidelines 
we  implemented  an  artificial  world  for  path  planning  (navigation).  This  artificial 
world  consists  of  linear  obstacles  of  random  length  and  orientation.  These  obstacles, 
called  "chaff"  hence  the  world  was  called  "Chaffworld,"  can  be  avoided  more  effi¬ 
ciently  by  planning  a  path  around  them  (deliberating)  rather  than  bumping  into  them 
(reacting),  but  this  efficiency  is  bought  at  the  cost  of  that  planning. 

Based  on  experiments  run  in  Chaffworld  (see  Section  3.2),  we  claim  that  the  design  of 
an  agent  must  reflect  the  task  it  undertakes  and  the  enviromnent  it  experiences.  In 
some  environments,  the  agent  will  be  reactive,  while  in  others  it  will  be  deliberative. 

In  environments  that  exhibit  qualities  of  both  reactive  and  deliberative  environments, 
the  agent  design  may  well  be  a  "two-loop”  design  (as  in  Phoenix).  We  show  that  we 
can  design  the  right  kind  of  agent  for  its  environment  by  knowing  the  cost/benefit 
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tradeoff  in  the  task,  the  level  of  abstraction  of  the  representation  used  in  deliberation, 
and  the  entropy  of  the  environment. 

2.3  Fo]]ow>iq>oii  Previous  ONRapoiiscxredReaeavch 

We  have  pursued  several  follow-up  activities  to  our  previous  ONR  funded  research 
and  take  this  opportunity  to  report  on  those.  We  specifically  report  continued  progress 
in  two  areas: 

A  Textbook  fiir  Empirical  Methods  in  Artificial  Intelligenoe.  We  have  been  preparing 
a  textbook  based  on  the  methodology  we  have  developed  for  conducting  research  in  AI, 
and  are  using  that  text  for  a  graduate  seminar  this  fall.  A  prospectus  of  the  textbook 
appears  in  Appendix  A. 

An  Automated  Model-Building  Assistant.  We  are  developing  an  automated  model- 
building  aseiatant  capable  of  extracting  causal  relationships  from  data  and  con¬ 
structing  analytical  models  to  guide  the  design  of  autonomous  agents.  The  long-tenn 
goal  of  this  research  is  to  embed  this  system  in  an  autonomous  agent  architecture, 
creating  agents  capable  of  integrating  perceptions  of  their  task  environments  into 
conceptual  models  that  support  reasoning  and  planning. 
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3  Detailed  Summary  of  Technical  ResuHs 


3.1  ^qdyingilie  Ccmtaici]MBgra<ionP<^iGy  to  Phoenix 
3J^  Specification  of  metaiM  of  perihrmanoe. 


In  the  grasping  problem  discussed  in  (Grupen  1991,  Grupen  &  Weiss  1991),  metrics  of 
performance  that  express  context  dependent  subgoals  of  the  task  (grasp  formation) 
are: 


*  Null  space  error.  This  term  expresses  the  error  of  the  contact  system  with 
respect  to  the  task  and  is  unimodal. 

*  Manipulability  of  the  hand.  A  convex  composition  of  the  manipulability  of  the 
fingers,  this  term  expresses  the  kinematic  quality  of  the  grasp  configuration. 

Admissible  and  predictable  controllers  for  grasp  optimization  are  constructed  firom 
the  above  mentioned  formal  metrics.  Cooperative  reactivity  emerges  firom  the  con¬ 
straints  imposed  by  the  manipulability  of  the  hand  on  migrating  contacts. 


In  Phoenix,  no  suitable  analogs  to  these  fmmal  metrics  of  performance  exist  cur¬ 
rently,  nor  were  we  able  to  construct  any.  Predictable  and  admissible  controllers 
constructed  firom  formal  metric  specifications  are  prerequisites  to  cooperative  reac¬ 
tivity  using  the  contact  migration  policy. 

3,1J2  ProoeasMioddiiiip 


In  the  grasping  problem,  uncertainty  arises  due  to  incomplete  information  about  the 
object  (to  be  grasped).  Grupen's  strategy  for  grasp  formation  refines  an  incomplete 
process  model  firom  local  information  obtained  at  individual  contact  sites.  This 
infisrmation  is  used  to  perform  reactive  adjustments,  to  optimize  the  grasp  locally. 
However,  the  object  to  be  grasped  is  rigid  and  static.  The  kinematics  of  the  hand  are 
fixed.  Further,  contact  sites  provide  sufficient  process  sampling  density.  These  rela¬ 
tive  advantages  simplify  the  construction  of  a  stationary  process  model. 

In  contrast,  parameters  of  fire  dynamics  change  continuously,  in  highly  nonlinear 
ways  as  fire  consumes  forest,  traversing  varying  terrain  and  subject  to  ^banging 
wind  velocity.  This  entails  the  construction  of  a  non-stationary  and  dynamic  model. 
This  a  harder  problem  and  requires  a  much  more  densely  sampled  process  than  that 
required  in  grasping  a  rigid  body.  Phoenix  observers  (i.e.,  the  bulldozers  fitting  the 
fire)  are  however  inherently  sparse.  We  have  determined  that  the  process  modeling 
problem  in  Phoenix  requires  a  different  approach  than  that  used  in  contact  migration 
-  one  sufficiently  different  that  the  contact  migration  algorithm  is  no  longer 
applicable. 


3.13  Reactivity  in  Fboenix 

The  Phoenix  environment  has  been  characterized  as  unpredictable,  dynamic  and 
complex  (Cohen  et  al.  1989).  Reactive  schemes  have  been  employed  in  unpredictable 
environments,  such  as  in  mobile  robot  navigation  discussed  in  (Arkin  1987,  Arkin 
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1988),  (Pajrton  1986,  Pcqrtom  1990)  and  control  theoretic  approaches  have  been  employed 
to  construct  models  for  reactive  interaction  between  the  agent  and  the  environment 
(Khatib  1985).  However,  purely  reactive  schemes  have  been  observed  to  foil  in  complex 
environments  (Ravela  1992),  whereas  global  planning  approaches  seem  to  be  more 
suitable  (Chrisman  1991).  Indeed,  planning  has  been  integrated  with  reactivity  to 
tackle  complex,  imcertain  environments;  mechanisms  for  integrating  planning  and 
reactivity  have  been  discussed  in  (Payton  1990;  Kaebling  1989;  Anderson,  Hart  & 

Cohen  1991;  see  below). 

An  understanding  of  the  modeling  problem  in  Phoenix  suggests  that,  though  imcer- 
tainties  may  warrant  a  reactive  component,  process  modeling  mxist  however  evolve 
from  predictive  capabilities  (expressed  as  contingency  anticipation),  domain  knowl¬ 
edge  and  map-based  reasoning  (expressed  as  plans  or  expectations),  and  incremental 
evidence  assimilated  from  reactive  behavior.  Chrisman  (1991)  describes  a  similar 
characterization.  Further,  the  spedfrcation  of  tasks  and  plans,  the  construction  of 
consistent  world  models  and  models  of  interaction  between  the  agent  and  the  envi¬ 
ronment  are  all  issues  that  form  the  basis  of  a  design  methodology  for  cooperative 
reactivity. 

3.1w4  Conclusions 

Simultaneous  effects  of,  limited  observability,  complex,  non-stationary,  nonlinear 
process  djmamics  and  the  presence  sparse  observers  inhibit  cooperative  reactivity  (as 
defined  for  the  contact  migration  policy)  in  Phoenix.  The  Phoenix  domain  represents 
a  dramatically  more  difficult  problem  for  the  behavioral  design  paradigm  than  that 
for  which  the  contact  migration  policy  was  developed.  Significant  developments  m 
modeling  coupled  with  knowledge/map  based  reasoning  and  heuristic  models  of 
weather  varUfoility  are  prerequisites  to  developing  cooperative  reactivity. 

3.L5  Siirv<ey  and  Graduate  Seminar  on  Reactivity 

Preliminary  to  the  above  investigations  we  conducted  a  broad  survey  of  the  application 
of  reactivity  in  robotic  systems  (Ravela  1992).  This  survey  provided  a  context  for  these 
investigations  and  is  also  the  basis  for  a  recently  developed  graduate  seminar  entitled 
"Reactive  Robotic  Systems"  being  offered  in  the  foil  of  1992  for  the  first  time.  The 
seminar  description  reads: 

This  seminar  will  focus  on  the  specifications  for  and  application  of  reactive  subsys¬ 
tems  to  integrated  robotic  systems.  We  will  review  recent  work  in  formal  specifica¬ 
tions  for  reactive  systems  which  yield  provably  convergent  controllers.  The  focus  will 
be  <m  sensor-based  navigation  functions  for  mobile  platforms  and  on  the  control  com¬ 
position  problem  for  high  dimensional  systems  with  excess  degrees  of  freedom.  The 
mobile  platform  study  will  deal  with  issues  of  dimensionality,  and  the  funon  of 
information  firom  vision,  sonar  and  touch  into  a  consistent  environmental  model.  We 
will  study  the  application  of  new  techniques  for  reactive  re-planning  in  this  context. 

The  redundant  systems  study  will  focus  on  issues  related  to  building  controllers  for  the 
Utah/Mrr  dextrous  hand.  The  emphasis  here  is  the  decomposition  of  hi^  dimen¬ 
sional  state  spaces  into  tractable  subspaces  and  the  dynamic,  on-line  re-composition  of 
controllers  mediated  by  sensory  information  and  the  task.  We  will  study  methods  for 
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building  structure  into  the  re-composition  problem  using  domain  knowledge,  and 
learning  composition  policies. 

3i2  Renolvfaig  the  DeKberative/Reactive  Dilftinma 

One  the  central  dilemmae  in  AI  in  the  past  several  years  has  been  deliberative 
planning  versus  reactive  planning.  The  former  is  what  AI  has  called  planning  since 
the  time  of  STRIPS,  while  the  latter  is  a  reaction  to  the  computational  complexity  of 
formal  planning  and  emphasizes  algorithms  that  simply  react  to  the  current 
situation. 

From  the  beginning  of  the  Phoenix  project,  we  saw  a  need  for  both  kinds  of  capability: 
the  foresight  of  a  deliberative  planner  in  order  to  coordinate  the  resources  of  multiple 
firefighting  field  agents,  and  the  speed  of  a  reactive  planner  in  order  to  smwive  sud¬ 
den  dianges  in  the  direction  and  location  of  the  fire.  Consequently,  we  designed  what 
we  have  called  a  "two-loop"  architecture  (see  Figure  1)  where  the  outer  sense-act  loop 
is  slow  enough  to  accommodate  deliberative  planning  while  the  inner  sense-act  loop  is 
quick  enough  to  react  to  a  changing  environment. 

In  our  initial  investigations  of  this  design's  efficacy,  we  derived  a  simple  mathemati¬ 
cal  model  of  the  interaction  of  two  tasks  running  concurrently  on  a  uniprocessor 
(Anderson,  Hart  &  Cohen  1991).  Specifically,  we  modeled  the  rate  at  which  two  peri¬ 
odic  tasks  would  interrupt  each  other.  The  predictions  of  this  model  were  supported 
by  simulation  experiments.  From  the  model,  we  argued  that  placing  reflexive  control 
and  cognitive  control  on  different  processors  is  not  justified  by  the  different  time- 
scales  over  which  they  act,  but  ts  justified  by  the  extent  to  which  the  higher  priority 
task  dominates  the  uniprocessor. 

The  research  question  we've  been  investigating  since  then  is  essentially:  when  should 
the  agent  deliberate  and  when  should  it  reactl  What  are  the  dimensions  of  that  deci¬ 
sion?  We  believe  the  answer  rests  on  the  following  principles  and  criteria: 

•  Deliberation,  especially  Torward  Search,"  is  computationally  expensive 

•  Deliberation  is  sometimes  useful  or  even  necessary 

•  Its  use  is  dictated  by: 

-  Computational  Cost  vs  Execution  Benefit 

-  Abstraction  Level 

-  Uncertainty 

*  Intrinsic  unpredictability  of  the  world 

*  How  must  precision  does  the  agent  demand? 
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Figure  1.  The  upper  picture  views  agent  behavior  very  abstractly  as  a  mapping  from 
perceptions  to  articulator  titgectories.  The  lower  picture  depicts  our  modularization  of 
that  abstract  mapping.  The  interface  between  the  modules  consists  of  trajectories  of 
subgoals  (subgoals  of  the  overall  behavior).  Adding  perceptual  input  to  the  interface 
allows  for  a  quick  feedback  loop,  thereby  allowing  reactivity. 

To  attack  the  question  of  cost  and  benefit,  we  implemented  an  artificial  world  for  path 
planning  (navigation).  This  artificial  world  consists  of  linear  obstacles  of  random 
length  and  orientation.  These  obstacles,  called  "diafT  hence  the  world  was  called 
"CShafiworid,"  can  be  avoided  more  efiSdently  by  planning  a  path  around  them 
{deliberating)  rather  than  btimping  into  them  {reactingX  but  this  efficient^  is  bought 
at  the  cost  of  that  planning.  An  idealized  chaff  encounter  is  depicted  in  Figure  2.  A 
typical  diafifworid  is  shown  in  Figure  3. 

We  used  an  off-the-shelf  A*  search  algorithm  to  do  the  planning,  rather  than  a  special 
purpose  algorithm,  so  that  the  results  will  not  be  specific  to  navigation,  but  be  more 
general.  The  mathematical  analysis  of  prototypical  encounters  (Figure  2)  suggested 
that  when  chaff  is  very  sparse,  planning  is  probably  not  worthwhile,  but  as  the 
density  increases,  the  advantage  of  planning  increases,  so  that  a  break-even  point 
may  be  readied.  This  break-even  point  is  the  thinking  ratio,  c  (measured  as  the  ratio 
of  the  map  area  it  can  search  per  unit  time  to  the  linear  distance  it  can  travel  in  unit 
time),  of  an  agent  that  is  indifferent  between  deliberating  and  reacting.  An  agent  with 
a  hi|^r  ratio  would  prefer  to  deliberate,  while  an  agent  with  a  lower  ratio  would 
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pre&r  to  just  start  moving.  Thus,  we  ha^  hard,  olgective  reasons  to  deliberate  or 
react 


ngureS.  An  agent  encounters  an  obstacle.  It  can  either  plan  its  way  around  (X?)  or 
bump  into  it  and  follow  it  around  (ABC). 


Figure  S.  The  scene  on  the  right  is  of  a  typical,  randomly  generated  Chaflworld.  The  scene 
on  the  left  is  the  agent's  map  of  the  Chafftrorld.  In  some  experiments,  this  map  was  identi¬ 
cal  to  the  world,  while  in  others,  shown  here,  short  pieces  of  chaff  are  not  represoited,  to  test 
the  hypothesis  that  abstraction  of  this  sort  will  improve  the  efficient  of  planning. 
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The  density  chaff  was  controlled  by  two  parameters  used  to  generate  the  world:  X 
the  mean  of  the  exponential  distribution  of  chaff  lengths  or  the  constant  length  of  all 
chaff,  depending  on  the  experiment;  and  p,  the  rate  of  the  Poisson  process  used  to 
scatter  the  chaff  across  the  world.  The  length  of  the  chaff,  X,  is  measiured  in  meters, 
while  the  rate,  p,  is  measured  in  pieces  of  chaff  per  square  kilometer.  The  experi¬ 
mental  and  theoretical  results  are  used  to  compute  Cmm  -  the  break-even  thinking 
ratio,  measured  in  square  meters  per  meter.  The  numerical  values  for  reasonable 
values  of  X  and  p  are  presented  in  Table  1.  The  experimental  results  are  mixed:  they 
do  suggest  tradeoffs  and  increasing  advantage  for  planning,  but  the  empirical  break¬ 
even  point  is  quite  different  firom  its  theoretical  value.  Further  experiments  are  being 
designed  to  explain  this  discrepancy.  One  hypothesis  is  that  the  experimental  situa¬ 
tion  (see  Figure  3)  cannot  be  adequately  modeled  as  a  random  iteration  of  the  idealized 
situation  (see  Figure  2),  because  a  locally  optimal  path  in  the  idealized  situation  is 
found,  while  the  A*  search  solves  the  experimental  situation  for  the  globally  optimal 
path.  Therefore,  the  experiment  should  be  redesigned  so  that  the  experimental  sitiia- 
tion  more  closely  corresponds  to  the  locally  optimal  solutions. 


Theoretical  Predictions  of  Cmin 


X 

0 

1000 

2000 

3000 

4000 

.1 

12,570 

7,260 

7,290 

6,410 

6,740 

4,490 

4,420 

.3 

4,900 

Experimental  Values  for 


X 

O 

1000 

2000 

3000 

4000 

.1 

224.1 

14.3 

19.6 

8.4 

.2 

47.3 

4.5 

4.3 

.3 

11.7 

Table  1.  In  this  experiment  and  theoretical  calculation,  chaff  was  assumed  to  be  dis¬ 
tributed  with  constant  length  X  and  density  p.  The  theoretical  and  experimental  results 
are  qualitatively  in  agreement:  smaller  X  or  smaller  p  produce  a  smaller  Cn,,,, .  Obvi¬ 
ously,  they  strongly  disagree  on  the  actual  value  of  e„/„ . 

Chaffworld  also  partially  addresses  the  question  of  abstraction  level.  The  abstraction 
level  is  the  primary  factor  in  assessing  the  cost  of  deliberation  in  the  form  of  search: 
Deliberation  will  always  cost  less  when  done  at  a  higher  abstraction  level.  However, 
the  disadvantage  of  higher  abstraction  levels  is  greater  error  in  the  planning,  hence 
less  benefit.  In  Chaffworld,  the  agent  planned  using  a  "map"  that  included  only  long 
chaff  (longer  than  some  threshold).  See  Figure  3.  Experiments  to  test  the  effects  of 
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abstraction  are  being  designed.  We  also  are  designing  experiments  that  will  vary  the 
abstraction  level  of  the  A*  search  by  using  finer  and  coarser  step  sizes  in  the  represen¬ 
tation. 

Finally,  we  are  addressing  the  question  of  the  effect  of  uncertainty  on  the  tradeoff 
between  deliberating  and  reacting  by  designing  an  agent  that  is  sensitive  to  the  imcer- 
tainty  of  its  environment  and  deliberates  until  the  acciunulated  uncertainty  makes 
fiurther  deliberation  futile.  We  believe  that  such  an  agent  will  act  effectively  in  both 
highly  uncertain  (high  entropy)  and  less  uncertain  (low  entropy)  worlds.  Further¬ 
more,  we  believe  that  the  agent  will  naturally  act  in  a  deliberative  manner  in  low- 
entropy  worlds  and  reactively  in  high-entropy  worlds. 

Essentially,  we  claim  that  the  design  of  an  agent  must  reflect  the  task  it  undertakes 
and  the  environment  it  experiences.  In  some  environments,  the  agent  will  be  reac¬ 
tive,  while  in  others  it  will  be  deliberative.  In  environments  that  exhibit  qualities  of 
both  reactive  and  deliberative  environments,  the  agent  design  may  well  be  a  ’’two- 
loop"  design.  We  can  design  the  right  kind  of  agent  for  its  environment  by  knowing 
the  cost/benefit  tradeoff  in  the  task,  the  level  of  abstraction  of  the  representation  used 
in  deliberation,  and  the  entropy  of  the  environment. 

3^  Follow-up  on  Previous  QNRrsponsored  Research 

3.3.1  A  Textbook  fiur  Empirical  Methods  in  Artificial  Intelligenoew 
In  the  final  report  for  our  previous  ONR  contract  we  detailed  several  ongoing  activities 
to  broaden  and  disseminate  our  methodological  approach:  a  Workshop  on  AI  Method¬ 
ology  held  in  June  1991,  the  development  of  a  curriculum  in  agentology  during  a 
modeling  summer  school  for  graduate  students  in  1991,  and  a  AAAI-92  Tutorial  on 
Experimental  Methods  for  AI  Research.  These  activities  are  culminating  now  in  a 
textbook  being  prepared  for  use  in  graduate  AI  methods  courses.  Entitled  "Empirical 
Methods  in  Artificial  Intelligence,"  this  textbook  is  a  primer  for  the  empirical  evalua¬ 
tion  of  the  new  generation  of  agents  being  designed  by  AI  researchers. 

3.3J2  An  Automated  Mbdd-Building  Assistant. 

Under  our  previous  ONR  contract  (N00014-88-K-0009)  we  developed  the  Modeling, 
Analysis  and  Design  (MAD)  methodology  (Cohen  1991),  an  approach  to  the  design  of 
autonomous  agents  that  combines  analytical  modeling  with  empirical  substantiation 
to  justify  the  agent  architectures  we  use.  We  are  employing  this  approach  in  the  con¬ 
text  of  the  Phoenix  system,  a  complex,  real-time  task  environment  in  which 
autonomous  agents  fight  simulated  forest-fires  (Cohen  et  al.  1989,  Howe,  Hart  & 

Cohen  1990).  T^th  support  from  a  recently  completed  URI  award  (N00014-86-K-0764)  we 
developed  a  number  of  tools  that  support  model-building  and  analysis,  and  have  used 
these  tools  extensively  in  Phoenix  (Cohen  1990a,  Howe  &  Cohen  1991)  and  more 
recently  in  the  domain  of  transportation  planning.  We  are  working  to  develop  an 
automated  model-building  assistant  ce  pable  of  extracting  causal  relationships  from 
data  and  constructing  analytical  models  to  guide  the  design  process.  The  long-term 
goal  of  this  research  is  to  embed  this  system  in  an  autonomoiis  agent  architecture, 
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creating  agents  capable  of  integrating  perceptions  of  their  task  environments  into 
conceptual  models  that  support  reasoning  and  planning.  This  work,  outlined  below, 
is  described  in  more  detail  in  (Silvey,  Loiselle  &  Cohen,  1992). 

The  primaiy  objective  of  the  IGOR^  project  is  to  concentrate  on  understanding  and 
automating  the  expert  human  process  of  incrementally  building  models  from  both 
experimental  and  non-experimental  data.  As  such,  this  work  represents  the  devel¬ 
opment  of  an  enabling  technology  for  principled  AI  research  using  the  MAD  method¬ 
ology.  IGOR  is  designed  to  function  as  a  model-builder's  intelligent  assistant.  The 
basic  approach  is  to  integrate  the  complementary  strategies  of  exploratory  and  con¬ 
firmatory  data  analysis  (eg.,  (Tukey  1977,  Wickens  1989))  in  a  knowledge-based  deci¬ 
sion  aid.  A  comparable  approach  which  integrates  complementary  data-driven  and 
theory-driven  strategies  is  described  in  (Langley  et  al.  1987). 

In  a  broad  context,  model-building  is  the  process  of  using  observations  of  a  function¬ 
ing  system  to  infer  stable  relationships  between  elements  of  that  system.  The  inferred 
relationships  facilitate  prediction,  explanation,  diagnosis,  and  planning  tasks;  and 
they  may  be  causal,  qualitative,  and/or  quantitative.  Models  are  built  by  agents,  both 
artificial  and  human,  to  summarize,  abstract,  and  generalize  from  experience.  Since 
models  represent  knowledge,  model-building  is  the  process  of  learning  or  acquiring 
knowledge.  Mod-els  are  constructed  and  refined  through  a  process  of  induction  from 
observation;  observation  which  may  be  active  as  well  as  passive.  For  example,  the 
environment  which  contains  or  comprises  the  system  in  question  is  often  occupied  by 
the  model-btiilder  as  well,  and  actions  taken  by  that  agent  will  themselves  affect  the 
problem  environment.  In  many  such  cases,  the  resulting  models  provide  a  subse¬ 
quent  means  for  the  agent  to  intelligently  control  the  environment,  as  they  consist  of 
knowledge  of  the  expected  effects  of  agent  actions  in  particular  situations. 

As  hiunans,  the  models  we  construct  firom  experience  in  our  environment  are  based 
upon  recognizable  dependencies  between  a  small  number  of  observable  variables. 
From  this,  we  are  able  to  construct  causal  explanations  of  how  the  system  in  which 
we  are  embedded,  fimctions.  The  specificity,  or  level  of  abstraction,  of  a  model  can 
range  firom  fully  quantitative  functional  relationships,  such  as  physical  laws,  to  prob¬ 
abilistic  or  qualitative  influence  relationships,  such  as  oiir  intmtive  understanding  of 
economics.  The  abstract,  higher  level  models  are  beneficial  because  they  provide 
interaction  constraints  that  hold  for  all  more  specific,  lower  level  models.  These  con¬ 
straints  can  reduce  the  search  reqxiired  to  find  spedfic,  quantitative  models,  because 
much  of  the  possible  search  space  is  ruled-out  by  the  general  patterns  of  dependencies 
in  observed  data. 

The  basis  of  constructing  causal  models  firom  observed  data  is  in  our  ability  to  detect 
conditional  independency  relationships.  The  methods  we  are  using  in  IGOR  to  find 
causal  structure  in  data  are  based  on  algorithms  developed  by  Pearl  (Pearl  &  Verma 
1991,  Pearl  1988).  The  approach  is  to  first  use  tests  for  conditional  independence  to 

^  IGOR  is  so  luunsd  becauss  it  is  intended  to  be  the  MAD  Scientist's  assistant. 
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construct  the  adjacency  structure  of  the  model,  an  undirected  graph  in  which  the 
nodes  represent  observable  variables  and  arcs  represent  direct  dependency  relations. 
This  structiure  is  then  turned  into  a  hybrid  graph  that  describes  an  equivalence  class 
of  directed,  acyclic  graphs  (DAGs)  that  correspond  to  possible  causal  models.  This 
hybrid  graph  has  arcs  that  have  zero,  one,  or  two  arrowheads,  roughly  corresponding 
to  undetermined,  causal,  and  spurious  correlations,  respectively.  The  undetermined 
correlations  are  underconstrained  in  the  data,  their  directionality  could  go  either 
way,  depending  on  the  assignment  of  other  unconstrained  arcs.  The  spurious  corre¬ 
lations  represent  hidden  causes,  or  latent  variables  as  they  are  sometimes  called,  that 
simultaneously  affect  the  two  observed  variables. 

These  algorithms  are  based  on  the  fact  that  the  "true"  causal  model  which  governs 
the  observed  system  imposes  some  constraints  on  the  joint  probability  distribution  over 
the  observed  variables.  Therefore,  we  can  use  statistical  tests  on  data  samples  of  those 
observables  to  partially  reverse-engineer  the  process.  However,  in  general,  it  is  not 
possible  to  construct  a  single  causal  model  that  is  uniquely  determined  by  the  data. 
Additional,  background  knowledge  may  be  required  to  take  a  partial  model  and  com¬ 
plete  it  to  the  experimenter's  satisfaction.  This  might  include  adding  arrows  to 
unconstrained  arcs  so  as  to  agree  with  the  flow  of  time,  or  the  addition  of  specific 
latent  variables  that  are  known  to  the  experimenter  but  were  not  measured  in  the 
specific  data  set  being  analyzed.  Thus,  we  see  these  automated  methods  for  discover¬ 
ing  causal  relationships  as  one  tool,  albeit  a  powerful  one,  for  building  models  firom 
data. 

IGOR  is  being  designed  using  the  blackboard  paradigm,  where  the  shared  blackboard 
data  structure  holds  elements  of  the  developing  model,  and  the  individual  knowledge 
sources  operate  on  those  elements  to  create  terms,  hypothesize  relationships,  and  per- 
fbnn  statistical  tests.  The  long-term  objectives  for  IGOR  include  fully  automated 
model-building  and  discovery  mechanisms  driven  by  an  opportunistic  control  strat¬ 
egy.  We  expect  to  develop  the  automated  strategies  from  our  experience  with  the  sys¬ 
tem  as  a  manual  analysis  decision  aid,  letting  human  analysts  provide  the  initial 
reasoning  control  strategy. 
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4  Publications.  Presentations.  Reports 

Befieroed  Papers 

Anderson,  S.D.,  Hart,  O.M.  &  Cohen,  P.R.,  1991.  IVo  ways  to  act.  1991  AAAI  Spring 

Symposium  on  Integrated  Intelligent  Architectures.  Also  in  SIGART  Bulletin, 

2(4):  20-24. 

Silvey,  P  J!.,  Loiselle,  C.L.  &  Cohen,  P.R.  Intelligent  data  analysis.  To  appear  in 

AAAI92  Fall  Symposium  on  Intelligent  Scientific  Computation.  Cambridge,  MA. 

October,  1992. 

UnrefereedReportB  and  Articles 

Ravela,  Srinivas  S.,  1992.  A  survey  of  reactivity.  Technical  Report  #92-61,  Computer 

Sdence  Department,  University  of  Massachusetts,  Amherst. 

Bo(du  or  Parts  Tliereof 

Cohen,  P.R  Empirical  Methods  in  Artificial  Intelligence.  Textbook  in  preparation 

(see  Appendix  A). 

Invited  Presentations 

Paul  R  Cohen 

•  Panel  member,  "The  Future  of  Expert  Systems"  chaired  by  Dr.  Y.T.  Chen  of  NSF 
at  the  World  Congress  on  Expert  Systems,  Orlando,  Florida,  December  16-19, 
1991. 

•  Panel  member,  "The  Empirical  Evaluation  of  Planning  Systems:  Promises  and 
Pitfalls"  at  the  First  International  Conference  on  AI  Planning  Systems  at  the 
University  of  Maryland,  Jime  16, 1992. 

•  Panel  member,  "Planning  Under  Uncertainty,"  SIGMAN  Workshop  on 
Knowledge-Based  Production,  Planning,  Scheduling  and  Control,  Workshop 
Program,  Tenth  National  Conference  on  Artificial  Intelligence,  San  Jose,  CA, 
July  1992. 

•  "Methods  for  Studying  Agents:  General  Issues  and  Specific  Examples."  Invited 
talk  at  the  U.S.  Navy  Center  for  Applied  Research  in  Artificial  Intelligence, 
Naval  Research  Laboratory,  Washington,  DC.  April  6, 1992. 

•  "Three  Examples  of  Statistical  Modeling  of  an  AI  Program."  Invited  talk  at  the 
University  of  Texas,  Austin. 

•  "Methods  for  Agentology:  General  Concerns,  Specific  Examples."  Invited  talks 
at  Virginia  Polytechnic  Institute  and  the  University  of  West  Virginia. 

•  Invited  presentation  of  EKSL  research  at  the  Institute  for  Defense  Analysis's 
Information  and  Science  Technology  Advisory  Group  on  Simiilation  in 
Washington,  D.C.,  June  17-18  (Cohen  is  a  new  member  of  the  Advisory  group). 
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Contributed  PresentatioDs 
Paul  R  Cdien 

•  Welcoming  address  (untitled)  at  the  NSF/DARPA  Workshop  on  Artificial 
Intelligence  Methodology.  Northampton,  MA.  June  2,  1991. 

•  "Intelligent  Data  Analysis."  Contributed  presentation  to  he  given  at  the  AAAI92 
Fall  Symposium  on  Intelligent  Scientific  Computation.  Cambridge,  MA. 

October  23-25, 1992 
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5  Transitions  and  DoD  Interactions 


5.1 


I  Research  Presented  at  die  Naval  Research  Laboratcny 


Paul  R.  Cohen  was  invited  to  speak  at  the  U.S.  Navy  Center  for  Applied  Research  in 
Artificial  Intelligence  at  the  Naval  Research  Laboratory  in  April.  His  presentation, 
entitled  "Methods  for  Studying  Agents:  General  Issues  and  Specific  Examples,"  cov¬ 
ered  research  conducted  under  this  and  previous  ONR  contracts.  The  first  part  of  the 
talk  presented  the  methodological  approach  developed  under  our  previous  ONR  con¬ 
tract  The  second  part  described  the  application  of  these  methods  in  our  current 
research  (induding  an  example  firom  ChafEworld,  described  above). 


52  Tmniritinii  nf  Methndologicnl  Tnnlia  te  TIABPA  Plnwtiitig  Tnltintiim 

Since  the  end  of  the  previoiis  contract  we  have  refined  our  methodological  approach 
and  begun  to  operationalize  aspects  of  it  for  use  as  components  of  IGOR  (see  above).  We 
recently  proposed  to  develop  and  test  several  experiment  modules  for  the  DARPA 
Planning  Initiative  (PI)^.  It  is  difGcult  to  design  experiments  to  evaluate  planners' 
performanoe,  and  even  more  difficult  to  model  the  fiictors  that  affect  perfonnance. 
Under  this  proposal  we  would  develop  modules  to  facilitate  experiments  to  answer  the 
following  questions:  How  well  does  a  component  of  a  planning/scheduling  system 
work,  and  how  will  changing  the  task,  environment,  or  structure  of  the  component 
affect  its  performance?  We  will  begin  by  developing  five  modules,  with  more  antici¬ 
pated.  The  first  five  are  for: 

1.  inducing  causal  models  of  the  factors  that  affect  the  performance  of  plans 
and  planners 

2.  detecting  statistical  dependencies  between  actions  and  selecting  the  most 
likely  explanations  of  the  dependencies 

3.  inducing  Markov  state-space  models  of  the  behavior  of  planners  and  simu¬ 
lated  plans 

4.  inducing  acausal  functional  relationships  between  factors,  including  those 
that  affect  the  performance  of  plans  and  planners 

5.  assessing  nonparametric  confidence  intervals  around  variables  such  as 
plan  completion  time  and  number  of  units  delivered. 

We  must  stress  that  although  these  modules  all  have  mathematical  foimdations  in 
statistics  and  probability  theoiy,  we  are  proposing  fuU  software  packages,  not  merely 
a  handful  of  statistical  tests.  Experiment  modules  will  help  the  researcher  in  all 
phases  of  eq;)eriment  design,  data  collection,  exploratory  data  analysis  and  model¬ 
building.  Furthermore,  the  focus  of  our  modules  is  explanatory  models  of  AI  system 
perfimnance,  not  simple  statistical  hypothesis  testing. 


^  IMS  contract,  "Expcrimontal  Methods  for  Evaluating  Planning  Systems,"  will  be  ftmded  to  start  in 
FY93.  It  will  involve  joint  work  with  reseraehsrs  at  Colorado  State  Unviersity. 
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Once  the  theoretical  underpinnings  of  these  modules  are  worked  out,  we  will  test 
them  in  an  extensive  set  of  experiments  with  two  systems,  the  Phoenix  planner  and 
TPSS,  the  Transportation  Plan  Steering  System  that  we  are  developing  for  the  PI.  The 
experiments  will  be  sufficiently  extensive,  thorough  and  documented  to  serve  as  case 
studies  of  the  applications  of  our  experiment  modules,  and  will  encourage  other 
researdiers  in  the  PI  to  adopt  the  modules.  In  addition,  the  modules  will  be  made 
available  as  part  of  the  Planning  Initiative's  Common  Prototyping  Environment 
(CPE),  fully  integrated  with  the  UMass  CLASP^  statistical  package  that  is  part  of  CPE. 


^  Devalopsd  unckr  URI  contract  #N00014-86<K*0764. 
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Appendix  A:  Textbook  Prospectus 

Empirkad  Methods  in  Artificial  IntelUgenoe 
Paul  R.  Cohen 


L  Introduction 

A.  AI  Programs  as  Objects  of  Empirical  Studies 

B.  Versions  of  the  Three  Basic  Research  Questions 

C.  A  Methodology  for  Achieving  General  Answers  to  the  Basic  Research  Questions 

D.  Kinds  of  Empirical  Studies 

E.  Data  Analysis  for  the  Four  Classes  of  Studies 

F.  Empirical  AI  in  Context 
n.  Exploratory  Data  Analysis 

A.  Data 

A.1.  Scales  of  Data 

A.2.  Transforming  data 
A.3.  Measurement  Theory 

A. 4.  Causal  Models  of  Data  Values 

B.  Visualizing  and  Summarizing  Data 

B. l.  Exploring  One  Variable 
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C.  Joint  Distributions 
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C.2.  Contingency  Tables  for  More  than  Two  Variables 
C.3.  Statistics  for  Joint  Distributions  of  Categorical  Variables 
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C.4.  IHsualizing  Joint  Distributions  of  Two  Continuous  Variables 
C.4.1.  Finding  Hints  of  Causal  Relationships  in  Scatterplots 
C.4.2.  Point  Coloring  to  Find  Potential  Causal  Factors 

C. 6.  Statistics  for  Joint  Distributions  of  Two  Continuous  Variables 
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E.  Series  of  Categorical  Variables.  Behavior  Traces 
E.I.  Visualizing  Behavior  Traces 
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C.  Four  Spurious  Effects 

C. l.  Ceiling  and  Floor  effects 
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F.  Pilot  Experiments 
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H.  Summary 

IV.  Hypothesis  Testing  and  Estimation. 
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E.  Parameter  Estimation  and  Confidence  Intervals 
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E.2.  Confidence  intervals  for  when  is  unknown. 

E.3.  An  Application  of  Confidence  Intervals:  Error  Bars 
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F. l.  Keporting  Confidence  Inteivals,  When  and  Why. 

F.2.  Errors 
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G.  Appendix  1 
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A.  Monte  carlo 
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C.  Bootstrap 
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